Large-scale sophisticated linguistic monitoring

A key means of monitoring current and evolving events is through their linguistic signatures in text-based information sources (e.g, social media like Twitter, Facebook, and Reddit). Given the vast quantities of available data, reliable automated analysis is vital for assisting human intelligence in detecting credible threats. This is especially true given both (i) the complexity of information communicated via language, and (ii) the complexity of the human communication process. Currently, automated analysis software capable of operating over large-scale datasets in real time is limited in its ability to both (i) reliably detect sophisticated non-linguistic information communicated via the linguistic signal, and (ii) reliably decipher the true meaning underlying a particular linguistic signal. As one example, consider subtle linguistic cues that communicate bias: one post may describe a group as “freedom fighters” while another describes them as “terrorists”. The connotation of the first expression communicates something quite different from the second, indicating the writer’s bias for or against the group, which itself may indicate sympathies or group affiliations. The emerging field of computational sociolinguistics (Nguyen, Doğruöz, Rosé, & de Jong, 2016) tackles the automatic detection of this kind of non-linguistic information from the linguistic signal (e.g., connotations: Rashkin, Singh, & Choi, 2015; identity: Pearl & Steyvers, 2012; Pearl, Lu, & Haghighi, 2017; mental state: Pearl & Steyvers, 2010, 2013; Pearl & Enverga, 2015; perspectives: Hardisty, Boyd-Graber, & Resnik, 2010; Card, Boydstun, Gross, Resnik, & Smith, 2015). As another example, consider the complex reasoning process people use to understand language in context: if someone posts “oh yeah, i just want to murder that guy”, there are several possible interpretations. First, this post may be an example of hyperbole or exaggeration, with the writer negatively disposed towards the person in question but without plans to actually murder him. Second, this post may be an example of sarcasm or irony, with the writer positively inclined towards the person in question but using this expression to make a rhetorical point. Third, this post is literal truth and so is a legitimate death threat that should be monitored. Recent approaches in the field of computational pragmatics draw on shared context and human processes of social reasoning (Frank & Goodman, 2012; Goodman & Stuhlmüller, 2013; Goodman & Frank, 2016) to identify the true interpretation behind a particular linguistic expression (hyperbole: Kao, Wu, Bergen, & Goodman, 2014; irony: Kao & Goodman 2015; resolving ambiguity: Savinelli, Scontras, & Pearl, 2017; politeness: Yoon, Tessler, Goodman, & Frank, 2016; metaphor: Kao, Bergen, & Goodman, 2014; humor: Kao, Levy, & Goodman, 2015).